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ABSTRACT 

Multipath transport protocols like MPTCP transfer data across 
multiple routes in parallel and deliver it in order at the re¬ 
ceiver. When the delay on one or more of the paths is vari¬ 
able, as is commonly the case, out of order arrivals are fre¬ 
quent and head of line blocking leads to high latency. This 
is exacerbated when packet loss, which is also common with 
wireless links, is tackled using ARQ. This paper introduces 
Stochastic Earliest Delivery Path First (S-EDPF), a resilient 
low delay packet scheduler for multipath transport protocols. 
S-EDPF takes explicit account of the stochastic nature of 
paths and uses this to minimise in-order delivery delay. S- 
EDPF also takes account of FEC, jointly scheduling trans¬ 
mission of information and coded packets and in this way 
allows lossy links to reduce delay and improve resiliency, 
rather than degrading performance as usually occurs with 
existing multipath systems. We implement S-EDPF as a 
multi-platform application that does not require administra¬ 
tion privileges nor modifications to the operating system and 
has negligible impact on energy consumption. We present 
a thorough experimental evaluation in both controlled en¬ 
vironments and into the wild, revealing dramatic gains in 
delay performance compared to existing approaches. 

1. INTRODUCTION 

Current mobile communication devices embed multi¬ 
ple communication interfaces to access the Internet (e.g. 
HSPA, LTE and IEEE 802.11). Transporting data be¬ 
tween a source and destination in parallel along mul¬ 
tiple paths is wcll-recognised, at least in principle, as 
an effective means to improve performance, e.g. in¬ 
crease throughput m , resilience (if one path breaks, 
the connection can gracefully failover to the remaining 
paths) [4] , and load balancing [5] . A well-known exam¬ 
ple of a multipath transport protocol is Multipath TCP 
(MPTCP) [6], which extends the traditional single-path 
TCP (SPTCP) to stripe the data of a single connection 
across multiple routes or subpaths. The most popular 
Linux implementation of MPTCP supports two sched¬ 
ulers to assign packets into subpaths [7]: a round-robin 
(RR) scheduler which iterates over each subflow regard¬ 
less of their latency properties, and the lowest RTT 
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Figure 1: Multipath uploads (2.4-Ghz WiFi + 
LTE) in a home environment. Mean goodput 
and standard error at the top. Box and whiskers 
for packet delay at the bottom. Details in 

First (LowRTT) scheduler that gives priority to paths 
with lower round-trip times (RTT). Finally, similarly 
to SPTCP, packet loss recovery is achieved by means of 
an ARQ mechanism, i.e., feedback from the receiver is 
used to detect and retransmit lost packets. 

1.1 The head-of-line blocking problem 

Although a promising technology, building an effi¬ 
cient, practically usable, multipath transfer mechanism 
remains highly challenging. Indeed, issues that did not 
exist in the single-path context appear now in the mul¬ 
tipath paradigm, fostering a rich amount of research 
on packet scheduling HUE], loss recovery m, and rate 
and congestion control [5]. In this paper we design and 
prototype S-EDPF (Stochastic Earliest Delivery Path 
First), a low-delay packet scheduler for multipath trans¬ 
port protocols. Like MPTCP, we buffer packets at the 
receiver as they arrive until they can be delivered to the 
application in order. This causes an additional delay to 
the delivery of packets as these may have to await others 
that (i) arrive out of order (a frequent event with mul¬ 
tipath transporting), or (ii) have been lost and need to 
be recovered (e.g. by retransmission). This is known as 
head-of-line blocking (HOL) and its effects on the per- 
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Figure 2: Two packets are to be scheduled across 
two subpaths of identical average delay. A “de¬ 
terministic” scheduler may assign packet 1 to 
subpath 1 (variable delay) and packet 2 to sub¬ 
path 2 (fixed delay) forcing packet 2 to wait 
buffered for packet 1 during slow instantiations 
of subpath 1. In contrast, scheduling packet 1 
in subpath 2 would lead to no buffering delays. 

formance of MPTCP are illustrated by the experiments 
of Fig. [TJ These consist of a set of uplink transmissions 
from a laptop attached to a home-based WiFi network 
(in the 2.4Ghz band) and a 3G/4G dongle (with Me¬ 
teor, an Irish provider), and show how delay scales up 
to half a second with MPTCP in a real environment, 
an intolerable value for real-time applications. 

Out of order arrivals. An adequate scheduling 
of packets is of paramount importance to minimise re¬ 
ordering delay [7] . Besides the two aforementioned sched¬ 
ulers used in the Linux implementation of MPTCP, 
it is worth highlighting EDPF (Earliest Delivery Path 
First) [8], which assigns packets to the subpath with 
earliest expected arrival. If links are deterministic and 
there are no losses, EDPF is optimal. However, trans¬ 
mission rates and propagation delays are random in na¬ 
ture BIS, and taking decisions based only on averages 
renders suboptimal performance as Fig. [2] illustrates @ 
Indeed, Fig. [Q shows mild improvements of EDPF rela¬ 
tive to MPTCP in a real experimental setup. 

Losses. Another source for the HOL problem is 
losses because packets have to wait until these are recov¬ 
ered. ARQ is a well-tested mechanism to address losses, 
unbeatable when there is no cost for feedback (neither 
in overhead nor in time). However, high RTTs due to 
congestion or long distances can severely delay the re¬ 
ception of feedback information and thus the delivery 
of retransmitted packets. A wealth of approaches based 
on ARQ or Forward Error Correction (FEC) has been 
proposed in the past (see m and references therein). 
Recently, the application of network coding in trans¬ 
port protocols has shown a major impact on throughput 

lr rhe study in [10] with MPTCP and US mobile ISPs show 
that 20% of packets experience > 150ms of reordering delay. 



Figure 3: With block codes, the receiver buffer 
stores as many degrees of freedom (linearly inde¬ 
pendent coded packets) as the size of the block 
to decode and deliver data to the application. 

performance. Network coding was first integrated with 
TCP by Sundararajan et al. T2J- SlideOR |13| uses 
a sliding block mechanism. CoMP m is a multipath 
transport scheme where the rate of coded packets is con¬ 
trolled by a credit-based method. A Fountain (rateless) 
code is applied to MPTCP in [2j to alleviate bottleneck 
issues with heterogeneous links. The fundamental prob¬ 
lem of all these works, however, is that the throughput 
gains come at the cost of high delay because a full set 
of coded packets need to be received before starting the 
decoding process (see Fig. 0. For this reason, we do 
not consider block or rateless codes in this paper and 
propose instead a novel streaming coding technique. 

1.2 Our contributions 

Our main contributions are summarised as follows: 

• We design S-EDPF, a novel scheduler that assigns 
packets to network interfaces exploiting stochastic 
information of each subpath’s delays, in contrast 
to MPTCP’s LowRTT or EDPF which do not take 
such information into account, with the goal of 
minimising the impact of out of order arrivals in 
the presence of variable (random) delays. See 

• S-EDPF features a novel streaming coding scheme 
that uses lossy links to transmit redundant (coded) 
information. This mechanism helps us to improve 
delay performance dramatically when there are packet 
losses caused by wireless noise or interference, and 

to exploit bad-quality links that otherwise would 
drag down overall performance. See 

• We implement S-EDPF in a real prototype. Our 
prototype is implemented in userspace and works 
efficiently on multiple platforms (Linux, Android, 
*BSD, MAC OS X) without the requirement of 
administration (root) privileges. See 10 

• We evaluate the performance of our prototype thor¬ 
oughly in both controlled environments, with emu¬ 
lated conditions to evaluate its behavior, and into 
the wild , to illustrate the gains of S-EDPF in real 
environments with real applications. See 0 

Fig. [T] shows the performance of S-EDPF for 3 dif¬ 
ferent settings (details in the following) demonstrating 
how delay can be greatly improved in real environments. 
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2. MULTIPATH STOCHASTIC SCHEDULER 

S-EDPF is in charge of selecting which path and at 
what time to transmit each packet. In this section we 
assume lossless channels (the extension to lossy paths is 
considered in (J5J) and address the problem of scheduling 
transmissions with the goal of minimising the impact of 
out of order arrivals in the presence of random delays. 

2.1 Model Description 

We have to schedule /C = {1,2,...} packets across 
V = (1,..., P} subpaths with the goal of minimising 
in-order delivery delay. Assume that on each path the 
time at the sender is slotted, indexed by S = {1,2,...}, 
such that one packet can be transmitted in a slot. Let 
t P ' S denote the start time in seconds of slot s on path 
p. We do not assume that the slots on different paths 
are aligned or that slots have fixed duration, and so the 
link rate of each path may change over time due to the 
action of congestion control, for instance. The time at 
which a packet sent in slot s on path p arrives at the 
destination is a random variable a PtS , with a PyS > t p<s to 
respect causality. We make the following assumption: 

Assumption 1 (Reordering) . Consider two slots 
with indexes a and b on path p. When a < b then 

a < «p,b- In other words, there is no reordering of 
arrivals within the same path. 

While packet reordering can occur within a single path 
(e.g. due to sudden routing changes), it is usually rela¬ 
tively infrequent compared to out-of-order arrivals across 
paths [15], and so Assumption [T| is mild. It is impor¬ 
tant to stress that Assumption [l] applies only to packets 
sent on the same path. Packets sent on different paths 
may still arrive out of order if they experience different 
(random) delays. Assumption [l] implies that the delays 
experienced by packets on the same path are correlated 
and not i.i.d. To make this explicit, let random vari¬ 
able Ap iSiS+ i = a PiS+ i - a P}S > 0, s = 1,2,... and 
define A p ,o.i = a P .i- Then, 

S 

Up ,s — ^ ^ Ap,r—l,r (1) 

r—1 

where a p . ± can be computed as a p ,i = 0 p ,i + Tp,i/R p ,i, 
where 9 Pt i, L Pt i and B p \ are the propagation delay ex¬ 
perienced, the size in bits of the scheduled packet, and 
the access link bitrate on slot 1 and path p, respectively. 

To facilitate scheduling we make the following regu¬ 
larity assumption, 

Assumption 2. A p , s , s+ i is i.i.d., i.e. A p , s , s _|_i ~ A p . 

Note that this assumption may be relaxed, at the cost of 
increasing the complexity of the scheduler. In addition, 
we make the following assumptions. 

Assumption 3. A packet is transmitted in every slot. 


Assumption [3] ensures that throughput is maximised. 
Note that, by sacrificing throughput, lower delay might 
be achieved e.g. by sending packets only along the path 
with lowest delay. However, we leave investigation of 
this type of trade-off between throughput for delay to 
future work and instead focus on the use of coded pack¬ 
ets to trade off throughput for delay (see ([3]). 

Assumption 4. Delays are upper bounded by T p . 

Assumption 5. The slot duration on path p can be 
approximated as being constant, T p , over window T p . 

Assumptions [4] and [5] could also be relaxed at the cost 
of a higher scheduling complexity. 


2.2 Minimum Delay Packet Scheduling 

Let pk &V denote the path on which packet k G K, is 
transmitted, and let Sk G S denote the slot on path pk 
in which packet k is transmitted. Packet k is therefore 
transmitted at time t PktSk and the time at which packet 
k arrives is random variable a Pk}Sk . 

At the receiver we require in-order delivery of pack¬ 
ets arriving from multiple paths. To achieve this, the 
receiver maintains a reassembly buffer where out of or¬ 
der packets are held until they can be delivered to the 
application in order. The delivery time of packet k is 
therefore the random variable, 

Tfc = max{a Pl , Sl , a P2tS2 ,, a Pk , Sfc } (2) 

It will prove useful later to rewrite this expression equiv¬ 
alently as follows. Let K p>k := {?£ {1,..., k — l},p q = 
p} denote the set of packets sent on path p with indices 
lower than that of packet k and 

Yp^k := max{a p , S( : q G K p , k } (3) 

We then have that 


Yfc = max{Fi, fc ,..., Y P:k , a Pfc , Sfe } (4) 

Our aim is thus to schedule packet transmissions (i.e. 
to select path-slot pairs ( p k ,s k ), k = 1,2,...) so as to 
minimise the mean in-order delivery delay 

N 


D := lim 

N—¥ OO N 


[Y k ]-t Pk , a 


(5) 


k=1 


2.2.1 Low-complexity Scheduling 

However, finding the optimal schedule that minimises 
eq. (O is a complex combinatorial problem. Thus, our 
goal is to design a scheme that solves the problem with 
as low computational complexity as possible in order 
to support the high bitrates expected from a multipath 
protocol. We start with the following lemma. 

Lemma 1 . Suppose Assumption Q] holds. To min¬ 
imise delay, it is sufficient to consider situations where 
packets on the same path are transmitted in order of 
ascending index. 
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PROOF. See the appendix. □ 

Lemma Q] is intuitive and greatly reduces the set of 
packet schedules that need to be considered, collapsing 
this down from all combinations of scheduling orders to 
which subset of packets are sent on each path. 

Now, observe that Y Ptk does not depend on the in¬ 
dices of the packets in set K Ptk , but only on the slots 
in which the packets are sent. By Lemma [T| packets 
are transmitted in ascending order on a path and by 
Assumption [3] no slots are left unused. Hence, 

Y Pyk = max{a p , 5 , s = 1,2,..., s p ,A;} (6) 

where s p , k = maxjs, : q £ K p k }. Further, by Assump¬ 
tion 2] the path K delay has an upper bound T p and 
so older slots {s : t PtS + T p < t PtSpk } do not affect the 
value of Y P ' k . It is therefore sufficient to calculate the 
max over a finite window of slots, 

Lp,/r — max{o P)S , s — s_ p k ,..., Sp^/c} (7) 

with s pk := - 8 Pt k and <5 P)fc = s Pjfc - max{s : 

tp,s + T p < t PtSpk }. By Assumption [5] the slot dura¬ 
tion can be approximated as being constant, T p . Hence, 
for slots s Pj fc > [Tp/Tp] sufficiently far away from the 
starting slot we have S pk = S p := \T p /T p ~\. That is, 5 Ptk 
is constant and it is sufficient to calculate Y pk over a 
fixed window. This assumption helps us to manage the 
computational complexity of the scheduling algorithm 
but can be readily relaxed e.g using a Chernoff bound 
it can be shown that the probability that Y Ptk depends 
on events occurring prior to the fixed window is small 
for an appropriate choice of window size. 

Recalling random variables A Pt s p k ,s p k +j = a p,s p k +j~~ 
a P: s p k , j = 0,1,..., S p , we can rewrite Y pk as 

Yp,k = a P,s p|fc + max {Ap,s p k ,s p ifc +i, • ■ ■, ^P,s p k ,s p k +s p } 

Sequence A(s p fc ) := {Ap^^+i, ..., Ap^^+jJ 
is, by Assumption^ i.i.d., i.e. A (s fc ) ~ A p . We have 
therefore arrived at the following result, 

Theorem 1 (Invariance of Y Ptk — a PySp k s p ). The 
distribution of Y p ^ k — a PtSp k -s p is invariant on path p 
for packets k scheduled in slots s P:k > S p = \T p /Tp\. 

That is, we can let Z p describe the distribution of 
eip^ Sp k —§ p ~ Z p V/c such that s p ^ k S p — [" T p /T p j. 
Theorem Q] helps us to greatly reduce the complexity of 
our scheduler as we don’t have to recompute Z p for 
every packet schedule. Moreover, note that the dis¬ 
tribution of a PySp k and Z p can usually be readily esti¬ 
mated in an online fashion. For example, the realisation 
of a PiSp k -s p may already be known via ACK feedback 
from the receiver. Otherwise, we can use past observa¬ 
tions of packet delivery times on path p to estimate the 
distribution of a PjSp k s p - We can also use past observa¬ 
tions to estimate the distribution of A p . To reduce the 


number of observations required to estimate the distri¬ 
bution and to reduce computational/memory cost, we 
can also use a parametric model and estimate the pa¬ 
rameters using past observations. For instance, when 
using a Gaussian model we can find a reasonably accu¬ 
rate approximation of Z p using (TB] . 

2.2.2 Stochastic Earliest Delivery Path First 

Using Theorem [TJ we can rewrite eq. (0 as 

1 N 

D = N lh ?L TV ^ E t max ^ 1 + “ tp*,**’- ■ ■ > 

00 k= 1 

Zp + ap t s P k -s P — t Pkj s k , T) Pk ,s k }} 

where D p s := a PtS —t PiS . By Assumption [3] and Lemma 
[TJ minimising D is equal to minimising 

1 N 

lim T7 EE[max{^i + a 1<s Sl 

iV-too iv *—' 
k =1 

Zp + ap tSp k -s P , Opfe.sfe}] 

and the minimum delay schedule is to greedily select 
( Pk,s k ) to minimise E[max{Zi + ai tSl k s 1 , ■ • ■ ,Z P + 
a P,s F k -s P ,a, Pkt s k }\- We thus propose Stochastic Earli¬ 
est Delivery Path First (S-EDPF) to schedule the trans¬ 
mission of packet k in the first available slot of such path 
p k with minimum expected reordering latency, i.e., 

pi = argminE[max{ Z\ + ai )Sl k s 1 , ■.., 

Pk 

Zp + ap }Spk -s P ,a Pk! s k }\ (8) 

3. LOW DELAY STREAMING CODE 

In the event of a loss, which is frequent in wireless 
links, the usual recovery method, ARQ, costs extra de¬ 
lay because of the round trip time that it takes for the 
repeat request to be delivered to the transmitter and 
the message to be retransmitted to the receiver. Neither 
LowRTT (MPTCP) nor EDPF use loss information to 
schedule non-retransmitted packets, though they both 
preferentially send data on links with higher rate. If 
these links were lossy, these scheduling choices could 
do more harm than good to the overall performance. 
Our approach is to precode enough information in coded 
packets to anticipate losses and help the receiver recover 
such packets without a need for retransmission. 

Similarly as above, we divide time into slots S = 
{1,2,...} each corresponding to the transmission of one 
packet. S-EDPF generates a coded packet c k and sched¬ 
ules it every r slots (which we refer to as the coding in¬ 
terval). A coded packet c k is a random linear combina¬ 
tion of information packets 1 to N T k , where N T = r — 1 
is the number of uncoded packets Ui sent between c k 
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Figure 4: Coding scheme. 

and Ck -1 across all subpaths, i.e., 


r — 1 information packets. Assume that each coded 
packet can help us to recover from one erasure, irre¬ 
spective of the subpath where the erasure has occurred. 
Denote by N T ^ p the number of packets sent in subpath p 
between two coded packets (including last coded packet 
if sent on p), i.e.. Ylpev ^ t ,p = T - Then, we have: 


N T -k 

Ck = fr { ui , U 2 , ■ ■ ■ ,«JV T -fc) '■= ^ W k j ■ Uj (9) 

3 =1 

with coefficients w k j , selected identically and indepen¬ 
dently, uniformly at random from a finite field of size 
Q. At the receiver, upon reception of c k , a matrix G k 
is constructed with rows formed from the coefficients of 
the received packets (normal uncoded packets adding a 
row with a 1 in the corresponding diagonal, all other 
entries being 0). Decoding can then be carried out on- 
the-fly using e.g. Gaussian elimination. In our analysis 
we will make the standing assumption that the field size 
Q is sufficiently large that with probability one each 
coded packet helps the receiver recover from one infor¬ 
mation packet erasure. That is, each coded packet row 
added to generator matrix G k increases the rank of G k 
by one. Note however that we evaluate a real decoder 
(with a non-null decoding failure probability) in [J5] 

In this way, the task of S-EDPF is to jointly sched¬ 
ule the transmission of coded and information packets 
leveraging the path selector proposed in fJU (that min¬ 
imises out-of-order arrivals). 

3.1 Buffering Delay Analysis 

In this section, we characterise the buffering delay 
performance of this coding scheme when delays are fixed 0 
Information packets are delivered to the application in 
order as they arrive (without buffering delay), until 
there is a loss. We name this state as “in-order deliv¬ 
ery” . When there is a loss, a “decoding process” starts: 
packets are buffered until the decoder has enough coded 
packets to fill the gaps (losses), at which point in-order 
delivery resumes. Following HU, we index each “in- 
order delivery” / “decoding process” period with j = 
1 , 2 ,..., and let the random variable Sj count the coded 
packets required in j to resume an in-order delivery 
state (Sj = 0 if stage j is already in the in-order deliv¬ 
ery state). The above is illustrated in Fig. [4] for r = 4 
and packets being received from P = 3 subpaths with 
erasure probability £i, £ 2 , and £ 3 , respectively. 

Theorem 2 (S process). Suppose we transmit in¬ 
formation packets across V = {1,...,P} independent 
subpaths with erasure probability {e ±,..., £p} and fixed 
delays. Suppose we insert a coded packet on path p c £ V 
(with erasure probability e Pc ) in between every N T = 

2 We assess this coding scheme in the presence of random de¬ 

lays experimentally in Sj5]and leave the mathematical anal¬ 
ysis for future work. 


1. For all £ p and N TtP such that A T ,p£p < 1, 

the mean of the probability distribution of S exists 
and is finite. 


2. The distribution of S is characterised by: 

P(S= 0) - (1 e Pc )^ T ’ Pc_1 (1 (10) 

i^Pc 

P(S= 1) = (N T , p -l)e Pc (l-e Pc ) N ^ [J (l_£ i )^ + 

jApc 


+ x] N T ,iCi( 1 - £i) jvr ’* 1 n(i - e j ) NT ' 3 

i^Pc 

( 11 ) 

P(S=k )» ^ye k (l-e) kN - (^ k ~_^ , Vfc > 1 

( 12 ) 


where e = 


c _ X^pg-p ■ 


N t 


3. The first and second moments of S can be approx¬ 
imated by the following closed-form expressions: 


E[S']«P(S'=1)4 


t(N t )c 2 ( l-e) N r 


1—re 


E[^]«P(5=l)+(l-^(l-re) 2 ) 


2 TN r e 2 (l-e) N - 


(1—re) 3 


(13) 

(14) 


T - Tlna'p Nr,p£p 

where e = —-. 

Proof. See the appendix. □ 


In this theorem we approximate the tail of the dis¬ 
tribution of S (see eq. (fl2l) l because the exact solu¬ 
tion requires running a complex iterative method to 
compute probabilities from a poisson-binomial distri¬ 
bution m- In more detail, we approximate a poisson- 
binomial distribution with a binomial distribution (see 
the proof of Theorem [2j2) which is more tractable, al¬ 
though not accurate in general. However, we find that it 
provides a good approximation of the complete distri¬ 
bution of S because we only apply it to its tail (i.e. 
in P(5 = k), \/k > 1), which does not contribute 
much to the whole distribution, i.e., Jf’k, L 2 = k) -C 

J2k- = ^)> m most of the cases of interest. We 
confirm its accuracy by means of simulations below. 

3.1.1 Buffering delay 

Under the assumption of fixed delays, we can now 
characterise the in-order delivery delay of an optimal 
earliest arrival delivery scheduler using Theorem [2] Let 
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Figure 5: Buffering delay. Simulation results 
vs. theoretical upper bound (Theorem [3]). 


us define a frame as the transmission of N T information 
packets plus a coded packet. The scheduler in each 
frame assigns N TtP packets to each subpath p such that 
r = ^2 pG -p N T ' P . The relationship between the S process 
and buffering delay due to losses is as follows. 


Theorem 3 (Buffering delay) . At the receiver, 
the asymptotic mean buffering delay per transmitted packet 
is upper bounded by 

2r(E[5] + p\s = 0)) ^ ^ {Nr ' p{Nr ’ p ~ 1} ’ 1} Ap 
time units. 

PROOF. See the appendix. □ 


To illustrate the above, consider a scenario with 2 
subpaths having equal rate and packet erasure probabil¬ 
ities ei and €2 = {0,0.2ei, 0.4ei, 0.6ei, 0.8ei, ei}, respec¬ 
tively. First, we compute the bound on packet buffering 
delay given by Theorem [3] for different coding inter¬ 
vals; second, we simulate these scenarios with a custom 
event-driven simulator and measure the mean buffering 
delay; and finally we compare both results in Fig. [5] 
From the figure, we conclude that Theorem [3] predicts 
buffering delay tightly and therefore it also helps to val¬ 
idate the approximations used in Theorem [2] 

3.2 Path selection for coded packets 

The above is a very useful tool for taking decisions as 
to when to schedule transmission of coded packets. In 
addition, the next result tells us how we can best exploit 
the degree of freedom given by multiple subpaths. 

Theorem 4 (Path choice for coded packets). 
Under the conditions of Theorem @ for a given N T}P , 
buffering delay is minimised when coded packets are sched¬ 
uled in such path p c with highest erasure probability. 

Proof. See the appendix. □ 


The importance of Theorem0]is considerable because 
it allows us to exploit bad-quality links to further im¬ 
prove overall performance and support a high degree 
of resiliency in our system. This contrasts to other 
approaches, such as MPTCP or EDPF, where multi- 
path transportation need not provide any gain in per¬ 
formance (even a loss in some cases) over its single-path 
counterpart when some of the subpaths are lossy [TO] . 

It can be easily shown that the delay gain from apply¬ 
ing Theorem |4] (as opposed to scheduling coded packets 
on the better fink) is a multiple of P(S = 1)t. The value 
of r is bounded by the capacity of the channel but, as 
r becomes larger, P(S = 1) also decreases because the 
decoding events are becoming longer. Our simulations 
(which we do not show here due to space constraints) 
also confirm this intuition and show that for small val¬ 
ues of r (far below the channel capacity) the gain in 
delay is of the same order of magnitude of the mean 
buffering delay; however, as the operating rate of the 
system gets closer to the channel capacity, the effect of 
the placement of the coded packet vanishes. 

4. PROTOTYPE IMPLEMENTATION 

We use the framework of Coded TCP (CTCP) PjJj 
to implement S-EDPF. CTCP is composed of a pair of 
SOCKS5 proxies that route data from/to TCP applica¬ 
tions into/from (multiple) UDP standard sockets. This 
gives us the ability to easily implement and evaluate 
congestion/rate control, coding/decoding and schedul¬ 
ing algorithms in userspace. This not only facilitates 
research and development but it also maximises its de¬ 
ployability: S-EDPF/CTCP runs in Linux, OS X, An¬ 
droid and *BSD without administration privileges. 

The encoder/decoder is implemented using finite field 
arithmetic in GF( 256). This lets us encode and de¬ 
code information efficiently, using XORs for addition 
and subtraction and quick table lookups using SIMD 
programming for multiplication, without compromising 
performance (i.e. with very low decoding error prob¬ 
ability). Decoding is based on a Gaussian elimination 
algorithm that is called for every received packet and 
a back substitution algorithm that is called when there 
are sufficient degrees of freedom (enough linearly in¬ 
dependent equations). Decoded packets are released to 
the TCP socket as soon as they can be handed in order. 

To assign packets into each subpath’s queue as de¬ 
scribed in © we leverage rate and delay information 
obtained from feedback to compute a Gaussian approx¬ 
imation of Z p every half a second using the “Partition 
MBT” algorithm proposed in m- Then, for every in¬ 
coming packet, we use the last computation of Z to 
solve eq. © and enqueue each packet accordingly. 

We also implement a simple Selective ARQ mecha¬ 
nism to assist our coding scheme. Received packets trig¬ 
ger the transmission of cumulative acknowledgements 
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Figure 6: Testbed configurations. 


(ACKs) that give the transmitter information regarding 
delivery delays (used by the packet scheduler), packet 
loss rate (used by the coding scheme), which packets 
have been lost (used by the ARQ mechanism), and con¬ 
gestion information (used for rate control). 

Finally, unless otherwise stated, we use CTCP’s de¬ 
fault congestion control algorithm, which is particularly 
suitable for the sort of networks we target (lossy/variable) 
and which is fair to other TCP flows (see BSD- 


5. PERFORMANCE EVALUATION 

In the sequel, we summarise a thorough experimen¬ 
tal evaluation in both controlled and real environments. 
First, we validate our prototype by comparing its per¬ 
formance with the Linux implementation of MPTCP; 
secondly, we run a set of experiments in controlled en¬ 
vironments emulating network conditions; and finally, 
we run an experimental campaign “in the wild”. 

5.1 Comparison With MPTCP 

Fig. m anticipated a comparison between MPTCP, 
EDPF and S-EDPF-r with different coding intervals r. 
This experiment was carried out in a home environment, 
uploading data for 30 s from a laptop attached to a 2.4- 
Ghz WiFi Access Point and a Meteor 3G/4G dongle to 
a remote server using iperf. For “Legacy MPTCP”, 
we used the Linux implementation of MPTCP v0.89 
with OL1A [T for congestion control and LowRTT as 
scheduler. We repeat the experiment 5 times for ev¬ 
ery scheme. The first conclusion is that MPTCP shows 
the worst performance in terms of both throughput and 
delay. This could be explained due to MPTCP us¬ 
ing a different congestion control scheme (less friendly 
to wireless losses as shown in im a packet scheduler 
(LowRTT) that does not account for delay variability 
(nor does EDPF), and the lack of FEC for packet loss 
control. The second conclusion is that S-EDPF (*) in¬ 
creases the capacity of the multipath network (due to 
a more efficient scheduling of transmissions) and (ii) 
is capable of trading data throughput for large delay 
improvements by tuning r (due to our coding scheme). 


/\ LowRTT (no coding) 


S-EDPF (no coding) 


EDPF (no coding) 


S-EDPF-16 



S-EDPF-8 

S-EDPF-4 




20 40 

Std. Dev OTT(1) (msec) 



Figure 7: Buffering delay and throughput upon 
random (controlled) delays and no loss. 

5.2 Experiments on a Controlled Environment 

We use tc/netem to generate normally distributed 
delay samples which are not correlatec(f| and dummynet 
to emulate link bitrates and drop packets randomly 
when required. Finally, to avoid external effects from 
congestion control (e.g. slow start), in this subsection 
we fix the contention window of each subflow to the 
bandwidth-delay product of each link (i.e. highest rate 
without causing congestion). Unless otherwise stated, 
we send data using iperf for 30 s from a laptop at¬ 
tached to two emulated networks, as depicted in Fig. 0 
and repeat each experiments 5 times per mechanism. In 
the sequel, we first evaluate one variable at a time (ran¬ 
domness, then losses) and then we test the performance 
of a real video streaming service. 


5.2.1 Random propagation delays 

We set up two links at 10 Mb/s each connected to 
a router. The router introduces 50 ms of fixed propa¬ 
gation delay in link 2 and a random propagation delay 
with mean 50 ms and variable standard deviation in link 
1. We don’t drop any packet in these experiments. We 
compare MPTCP’s LowRTT scheduling scheme, EDPF 
and S-EDPF-r (i.e. S-EDPF with coding interval r). 
Results are shown in Fig. [7] The top subplot shows 
the average goodput performance experienced at the 
application layer and the bottom subplot whiskers and 
boxes for the measured buffering delay (i.e. not includ¬ 
ing the link’s propagation delay). Note that we fully use 
the aggregate capacity of the network, i.e., the receiver 
always receives 20Mb/s of raw traffic. In this way, 

3 This is particularly hostile for S-EDPF whose design as¬ 
sumes that delays in the same path are correlated. 
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(a) Heterogeneous links. (b) Homogeneous links. 

Figure 8: Buffering delay and throughput upon 
fixed delays and controlled losses. 

this figure illustrates how our FEC mechanism uses part 
of this capacity to send redundant information; for in¬ 
stance, S-EDPF-4 trades 5 Mb/s to reduce delay by half 
(roughly) when the standard deviation of link l’s prop¬ 
agation delay is larger. Note that coding helps delay 
even when there is no loss. The reason is that depending 
on the behaviour of the channels, coded packets might 
arrive sooner than preceding information packets and 
thus help the receiver to decode still-on-the-air packets 
sooner than the case when there is no coding. We have 
evaluated this scenario with different mean delays and 
S-EDPF offers similar gains in performance (not shown 
here due to space constraints). 


5.2.2 Lossy subpaths 

We now evaluate the performance of S-EPDF in the 
presence of lossy paths. To this aim, we fix the propa¬ 
gation delay of two links to 50 ms and the bandwidth 
at 10 Mb/s, and randomly drop 10% of packets in link 
1. In link 2, we don’t drop any packets in the experi¬ 
ments depicted in Fig. [Sal and 10% of packets in Fig.lSbl 
Note that we only drop data packets and not acknowl¬ 
edgements, to benchmark against an ideal ARQ mecha¬ 
nism. In the figures, we compare our S-EDPF scheduler 
with different coding intervals and a scheme that only 
relies on retransmissions (ARQ) to handle losses, like 
MPTCP. We use EDPF to schedule packets in “ARQ”. 
As depicted in the figure, the scheduling of FEC coded 
packets helps us to practically eliminate buffering delay 
with the most aggressive configuration (S-EDPF-4) at 
the cost of 15% of throughput. It is worth mention¬ 
ing that higher RTTs show larger gains in performance 
(not shown here for space reasons) because ARQ only 
retransmits packets upon the reception of feedback and 
this takes longer the higher the propagation delay. 



Figure 9: Youtube video streaming. 

cations, we next evaluate the performance of S-EDPF 
with a popular video streaming service. We stream a 
video from Youtube to a vie video player and collect 
statistics of video frames displayed and dropped (e.g. 
due to excess buffering delays). We have selected an HD 
video (with resolution 1280x534), Star Wars VII teaser 
trailer 20 encoded with H264/MPEG-4 AVC, with du¬ 
ration of 1:49 min, and a rate of 23.97 fps. We configure 
the two access links with a bandwidth of 10 Mb/s, an 
RTT of 100 ms, and a variable random packet loss rate 
(for both data and ACKs) on one of the links; we don’t 
drop packets on the other link. We also set vie with 
150 ms of caching. Note that, although we emulate 
network conditions on the access links, we do not have 
control over the network between our proxy server and 
the Youtube server (i.e. Internet - see Fig 0. 

We stream the video using EDPF (i.e., relying only 
on ARQ) and S-EDPF-r for different coding intervals 
r, and plot in Fig. [9] the mean ratio of video frames that 
have not been displayed. We repeat each experiment 5 
times. The results show a dramatic improvement on the 
streaming experience when using S-EDPF. In particu¬ 
lar, when the access link experiences losses, the video 
delivered with EDPF stutters and skips significant sec¬ 
tions of the video on play back, as illustrated by the 
figure. In contrast, S-EDPF skips essentially no video 
frames and playout is consistently smooth. 

5.3 Complexity 

Given that our system adds an additional layer to the 
network stack (in user space), there is a risk of increas¬ 
ing the processing cost of the communication, and thus 
threatening the lifetime of battery-powered devices. We 
have performed a thorough profiling of our application^ 
which highlights the encoder/decoder as the most costly 
element, as expected, because it uses CPU-intensive op¬ 
erations. In order to evaluate its complexity, we have 
installed our prototype in an Android LG G5 smart¬ 
phone and we have measured the energy consumption 


5.2.3 Video Streaming 

In order to assess the impact of the throughput and 
delay performance gains observed above on real appli¬ 


4 https : //www. youtube . com/watch?v=wCc2v7izk8w ac¬ 
cessed on 26/5/2015. 

5 Using tools like valgrind/callgrind and a power meter. 
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Figure 10: Energy consumption and goodput of 
our prototype vs. legacy sockets. 

of the device using a MonsoorQ power meter. To simplify 
the setup, here we only use a local WiFi hotspot (i.e. a 
single path) with fixed rate (54 Mb/s), fixed CPU fre¬ 
quency, and backlight turned off. We send TCP traffic 
from the device to the AP using iperf for 30 s, repeat¬ 
ing each experiment 5 times and collecting the average 
throughput and power. In the AP, we randomly drop 
packets at different rates and compare, in Fig. [10l the 
performance of S-EDPF, S-EDPF without our coding 
scheme (i.e. only relying on ARQ) and legacy TCP. 
For S-EDPF, we send as many extra coded packets as 
needed to compensate for losses. The first conclusion 
drawn from Fig. |T0]is that, when there are no losses, the 
throughput provided by both S-EDPF and TCP is the 
same, though at an extra energy cost of ~30 mW. This 
represents a negligible ~5% of additional consumption 
(if we set the backlight with the highest level of bright¬ 
ness, this cost goes down to barely ~l-2%). When there 
are losses (not caused by congestion), the higher protec¬ 
tion that S-EDPF’s congestion control has versus legacy 
TCP’s renders higher throughput performanceQ Note 
that legacy TCP consumes considerably less energy sim¬ 
ply because the bitrate is much lower. It is also worth 
noting that the additional encoding performed by S- 
EDPF (as compared to “S-EDPF (no coding)”) does 
not seem to entail a significant energy burden. 

5.4 “Into The Wild” 

Finally, we assess the performance of S-EDPF under 
real conditions. To this end, we use a laptop attached 
to a 3G/4G dongle and connect to different public WiFi 
hotspots around the city of Dublin, Ireland, as depicted 
in Fig. [6] Our experimental campaign covers measure¬ 
ments in the campus of Trinity College Dublin, using 
a departmental WiFi network; a home environment, in 
a complex with many apartments; a public pub, dur- 

6 https://www.msoon.com/LabEquipment/PowerMonitor/ 

7 See [T9] for a formal study of S-EDPF’s congestion control. 




ing busy hours; St. Stephen’s Green Mall in Dublin 
during a weekend day; and Dublin airport. For each 
location, we download a large file for 30 s and repeat 
the measurement 5 times for each configuration: EDPF 
and S-EDPF-r with different coding intervals r. Fig, fill 
depicts the average goodput of the downloads and stan¬ 
dard errors at the top subplot, and box and whiskers 
to measure packet delivery delay in the bottom sub¬ 
plot. The results illustrate how S-EDPF always shows 
dramatic gains in delay performance, spanning from a 
mean delay reduction of 40% in campus to a surprising 
86 % in the airport when using r = 4. It is also worth¬ 
while mentioning that the throughput performance of 
EDPF in the “Airport” and “Home” tests does not im¬ 
prove over that when using only a single path (not 
shown here due to space constraints), a result that is 
consistent with that of m s. The reason is due to the 
excessive buffering delay caused by both the losses and 
high variability in propagation delays and access rates. 

In contrast, even in these hostile environments, S-EDPF 
successfully combined the capacity of both links while 
keeping the mean packet in-order delivery delay low. 

6. CONCLUSIONS 

Multipath transport protocols are a promising tech¬ 
nology to increase reliability and aggregate the capacity 
of multiple access providers. However, as we have illus¬ 
trated experimentally in this paper, schedulers that do 
not consider transport delay variability, like LowRTT 
(MPTCP) or EDPF, suffer from a severe performance 
degradation in terms of delay that make them unsuit¬ 
able for real-time applications. We have also shown that 
using ARQ to recover lost packets further penalizes de¬ 
lay when round-trip times are large due to congestion 
or distance. To combat these issues, we propose in this 
paper a mechanism named Stochastic Earliest Delivery 
Path First (S-EDPF) that jointly schedules information 
and redundant (FEC) packets across the multiple net¬ 
work interfaces to (i) minimise the impact of packet 
reordering at the receiver when links are variable, and 
(ii) account for low-delay packet error recovery. 
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APPENDIX 
A. PROOFS 

Proof Lemma [Q Let a Pi i, a p> 2 , ..., a Pi „ denote re¬ 
alisations of random variables a Pi i, a Pi 2 , • • •, a Pj „ i.e. the 
arrival times of packets sent in slot n of path p. 

Suppose packets on the same path are sent in as¬ 
cending index order, i.e. for any k G {l,s,...} and 
91,92 G K Ptk such that 91 < q 2 then s pm < s p ^ q2 . By 
Assumption [lj it follows that o. pm < a Pi(?2 . Let if Ptk := 
max{a PiSj : 9 G AT p , fc } and if k := max{^ lifc ,..., V’p.fc, a Pfc , Sfc } 
Then we have if Pyk < V’p.fc+i, k G {1,2,...}, with 
equality only when K pk+ 1 = IL p To see this, note 
that when K Pyk+ \ = K p ^ k equality is trivial, and when 
d^ P ,k +1 7 ^ K P ,k then q k ^\ •— max {9 G AT P) / C ^_i} A 
9 ^. := max{g G K p , k } due to the ascending order and 
so a p,<it + 1 > a p,q* k an d thus ipp,k +1 > i>p,k- Hence, 
tpk < ipk+ 1 , k G {l,s,...} i.e. the in-order delivery 
time is strictly increasing. 

Suppose now that the transmission slots of two pack¬ 
ets 91 and 92 with qi < q 2 on path p are swapped, so 
that the packets are now sent in non-ascending order 
s qi > s q2 . Let := max{a PiS? : 9 G K Pjk } and 

VT" “ maxW;,...,^".^^}. Then ^ < 
V’ p ,fe +1 but now for at least one k G { 1 , 2 ,...}, namely 
k = q 2 , there will be equality when K pk +i ^ K Pyk and 
so if k on = V’fe+i- Further, for k > q 2 then i/j k on = 
ipk and also for k < q\. Hence, ip k on > ^ k for k G 
{ 91 ,..., 92 — 1 } and so the sum-delay is increased rela¬ 
tive to sending packets in ascending order. 

We can proceed by induction to show that swapping 
further packets cannot decrease the sum-delay below 
that when packets are sent in ascending order (there 
are two cases to consider, (i) when the further set of 
swapped packets is disjoint from those already swapped, 
in which case the above argument can be re-applied di¬ 
rectly, and (ii) when the further set of swapped packets 
intersects with those already swapped, in which case we 
can recover an ascending packet order). 

Since the above holds for any realisation a p q ,..., a Pjn , 
we are done. □ 

Lemma 2 (Lyapunov Central Limit Theorem). 
Suppose { X \, X 2 , • • ■ } is a sequence of independent ran¬ 
dom, variables, each with finite expected value pi and 
variance of. Let us define = Y^i=i a i- k-f for some 
5 > 0 , the Lyapunov's condition 

1 " 
bn i= 1 

is satisfied, then a sum of (Xipi)/s n converges in dis¬ 
tribution to a standard normal random variable, as n 
goes to infinity: 

1 " 

— — pi) —> Af(0, 1 ). 


Proof Theorem [2j S process. The proof for 1)- 
4) goes as follows: 

1 . We know that for the decoding process to go be¬ 
yond k we need at least k erasures in the first k 
frames. This means that there are cases with more 
than k erasures in the first k intervals that the 
decoding process stops before k but for it to be 
greater than k we should have at least k erasures. 
Let E k denote the number of losses in k frames. 
We have P(S > k\E k < k) = 0. More formally: 

P{S >k) = P(E kl > k)P{S > k\E k i > k) 

+ P(E k < k)P(S > k\E k < k ) 

= P{E k > k)P{S > k\E k > k ) 

< P{E k > k) 

E k is the summation of r independent Bernoulli 
random variable with parameter a, in which i de¬ 
pends on the path. By Lemma [2] we know the 
Lyapunov's condition is satisfied and 

, 1 ={E k - ^2 kN T , p e p ) 

y Spgp kN T , p e p (l — e p ) p ^-p 

converges in distribution to Af(Q, 1 ) for large values 
of k. P{E k > k) goes to zero only if k is larger than 
IZpev kN T ’ P c p . This means 

'y ] X T p e p A 1. 

p&V 

2. It is trivial to compute P(S= k), VfcG{0,1} exactly. 
Now, following the proof of Theorem 1.2 in m , we 
can compute P(S = k), k > 1 as: 


P(S = k) = 

min (fc,Z) 


(15) 


r —1 


= Y^ P ( C ( e } = r> )]—[ P ( C ^x(k-i)= k ~r))yk > 1 


r—2 


where e = {e Pl ,..., e Px }, e Pi is the erasure prob¬ 
ability in the subpath assigned to packet i, and 
[ e ] x(fc_i) appends the e vector k — 1 times. How¬ 
ever, the computation of P(£(e) = r) requires a 
complex iterative algorithm [ 15] , which is not feasi¬ 
ble to implement in S-EDPF. We thus approximate 
this part of the S distribution with: 


P(S = k) » P(Si = k) = 

min(k,l ) 

r— 1 


(16) 


= ^P(H(e,r=r)) —P(S(e,r(fc- 1 ) = k-r)) 


> 1 


where B is the binomial distribution with parame- 

— _ 'Etp^'p N Tj p€p 


ters e = 


AG 


and N T = ^2 A TjP . 

pGV 
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3. Let us define Sj as a random variable with the dis¬ 
tribution of S but with erasure probability e = 
^ p£ a/ Vt ’ P6p for all subpaths. Following |T7] . the 
first moment and second moments of Si are 


E[Si] = 

E[S 2 ] = E[S] + 


l)e(l-e) 


1 -ALe 


Nt — 1 

(17) 

l)e 2 (l - e) Nr 

(18) 

$ 

At 

W 


Now, E[S] can be approximated as follows: 


E[S] ra P(S = 1) + kP ( S i = k )• ( 19 ) 

fe =2 


Note that E5*L 2 kP ( s i = k ) = E [-Sj] - -PC- 51 ! = !), 
Thus, 


E[S] » P(S = 1) + E[Si] - P(Si = 1) = 

/V T ( N t —1 ) e 2 (1—e) Nt ~ 1 


= P(S= 1) 


1-JVt 


( 20 ) 


given that P(Si = 1) = (JV T -l)e(l-e)^" 1 . E[S 2 ] 
can be approximated similarly. 


□ 

Proof Theorem [3j In order delivery delay . 
Let us assume that a transmission of a stream of Nt 
packets is approximately a multiple of r at time t. Upon 
this assumption, we have that, at time t , we have trans¬ 
mitted — frames. Let us assume now that the de- 

r 

coding process consists of decoding periods of length 
{si, S 2 ,..., s n ,... }. Note that {Si, S 2 ,..., S„} is se¬ 
quence of positive independent identically distributed 
random variables. Assume S + = min{S, 1}, and define 
J n as follows: J n = E"=i S t , n > 0; then the renewal 
interval [J n , J ra+ i] is a decoding period. Let (X t )i>o 
count the decoding r-intervals that have occurred by 
time t , which is given by 

OO 

X t := ^2 = SU P { n : J n < t } 

n= 1 

and is a renewal process (I is the indicator function). 

Let W±, W‘ 2 - ... be a sequence of i.i.d. random vari¬ 
ables denoting the sum of in order delivery delay in each 
decoding frame. We have two cases to consider. Case 
(i): suppose the j’th period is an idle period. Then 
Sj = 0 and the information packets are delivered in- 
order with delay A p , where A p is the transmission time 
of a packet sent in subpath p, here assumed as con¬ 
stant. Case (ii): suppose the j’th period is a busy 
period and the information packet erasure that initi¬ 
ated the busy period started in the first slot + 1. 
Then the first information packet in each path p is de¬ 
layed by SjA, p N T p slots, the second by SjA p N Tp — 1 
slots and so on. The sum-delay over all of the informa¬ 
tion packets over all different paths in the busy period 


is therefore EpeptEfclt^' 

S? A p N TiP (A p N T iP — 1) 


k ~ Zto 1 k \N T , p ) < 


Epg-p tL ' The random variable Y t = 

E£i Wi is a renewal-reward process and its expecta¬ 
tion is the sum of in order delivery delay over the time- 
span of t. Based on the elementary renewal theorem for 
renewal-reward processes [20] , we have: 


lim -E[Y t ] = . 

*-► °° t 1 1 E[Si~] 


( 21 ) 


Based on the construction of Wi, we have E[lLj] = 

E,=i E [^il^ + = i]Pr(S+ = i) = ESo E [^il^i = 
*] Pr(Si = i) = E[S 2 ] E peP A ”W P (A P W P -i) _ Wg alg0 


|E[y t ] < 
. Since we assumed 


have that E[S{~] = E[S]+P(S = 0), and that lim^oo t 
EftS 2 ] A p N TjP (A p N T]P — 1) 

E[S]+P(S=0) A, P ^V 2 

that Nt = tr, we have: 

r 1 prvi ^ Epgp \ n t,p(A p N t , p - l) 1prc2l 

ivUoo N t E ^ ^ ~ 2r(E[S] + P(S = 0)) ^ 

□ 


Proof Theorem [4j Path choice for coded packets. 
Suppose we have a system with V = {1 ,,p} subpaths 
with erasure probabilities ei > 62 > ■ ■ ■ > e p . Let us 
first evaluate two independent S processes, S 1 and Sj, 
with p c = 1 and p c = P 2 respectively, i.e., we schedule 
our coded packets on path 1 in the first case, and P 2 
(any other) second. We know from Theorem [2] that the 
path selected for the coded packets does not affect the 
decoding process (delay) when S > 1, i.e., 

P(Sj = k) = P{S 2 = k ), V/c > 1. (22) 


When EpeP N TiP e p < 1 (i.e. w hen we operate below 
the capacity limit), the above yields: 

P(Si = 0)+P(Si = 1)=P(S 2 = 0)+P(S 2 = 1) (23) 


given that Efc P(S = k) = 1. 

Note that the decoding delay is equal to zero slots 
with probability P(S = 0) (because it is the in-order 
delivery state), and non-zero otherwise (packets are be¬ 
ing buffered until all losses are recovered). This means 
that, given equations (l22l) and (l23l) . the Si process that 
achieves lower delay is the one that maximises P(S = 
0). Let us then compare P(Sj = 0) and P(S 2 = 0): 

P(Si = 0) (l- C i)^.i- 1 n i# i(l-6i) Jv ^ 

P(S 2 = 0) (1 - 6 P 2 ) n ^ 2 - 1 n i#P2 (l - 


_ (^ e P2 ) > 1 

(1-er) ~ 


(24) 


because 1 > ei > e P2 > 0 . Thus P(Sj = 0) > P(S 2 = 
0), i.e., scheduling coded packets on path 1 (the one 
with highest loss probability) minimises the decoding 
delay, and this is valid for any Si process different than 

Sj. □ 
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